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SUMMARY 


Human  performance  data  are  required  for  the  design  and  evaluation  of  flight  simulators. 
Tins  paper  desctibes  a  data  collection  method  which  enables  information  to  be  collected  on 
ihe  visual  behavior  of  pilots  during  flight  simulation  The  method  described  supplements 
traditional  data  collection  techniques  such  as  pilot  performance  measures  and  questionnaires. 

Eye  position  data  are  collected  using  a  head-mounted  eye  tracking  device  which  incorporates 
infrared  sensors  to  detect  eye  movement.  A  head-mounted  camera  displays  the  viewed  scene 
and  a  superimposed  eye  position  locator  onto  a  video  screen.  A  video  recorder  is  used  in 
conjunction  with  a  time  code  generator  to  record  the  data.  These  data  are  then  reduced  using 
a  computer  software  program  that  summarizes  the  amount  of  time  and  the  number  of  glances 
for  particular  areas  within  the  scene. 

This  data  collection  method  was  used  to  gather  information  on  the  field-of-view  (FOV) 
requirements  for  the  C-130  Weapon  System  Trainer  at  Little  Rock  APB,  AK.  Eye  position  data 
for  twelve  C-130  pilots  were  monitored  to  determine  the  effects  of  a  full  FOV  versus  a  limited 
FOV  during  a  low-level  flight  and  an  airdrop.  Pilot  performance  measures  indicated  no  significant 
effects,  but  effects  were  significant  for  the  eye  position  data.  The  eye  position  effects  indicated 
that  pilots  used  a  different  visual  strategy  in  the  two  conditions.  These  data  show  how 
experienced  pilots  adapt  to  varying  FOV  configurations  to  maintain  flight  parameters.  The  results 
demonstrated  the  usefulness  of  the  eye  tracking  system  when  used  in  conjunction  with  traditional 
data  collection  methods. 

A  second  study  was  performed  in  which  the  eye  tracking  system  was  employed  to  determine 
FOV  usage  in  the  air-to-air  environment.  Twelve  F-15  and  F-16  instructor  pilots  performed 
air  to  air  maneuvers  for  various  training  setups.  Data  analysis  is  currently  undcavay.  but  Initial 
results  indicate  that  the  system  can  record  eye  position  data  in  the  dynamic  air-to  air  environment 

The  major  advantage  of  the  eye  tracking  approach  is  that  it  provides  a  direct  measure  of 
the  pilot's  eye  position  without  restricting  head  movement.  The  absence  of  a  fixed  reference 
point  and  the  labor-intensive  data  reduction  process  are  two  major  limitations  which  are  currently 
being  addressed 


PREFACE 


This  paper  was  written  in  support  of  the  Air  Force  Human  Resources  Laboratory's  current 
Research  and  Technology  Plan.  The  Training  Technology  goal  for  the  Operations  Training 
Div:sion  is  fo  develop  and  maintain  enhanced  job  perfoimance  and  combat  readiness  by 
id;  ntifying  and  demonstrating  cost-effective  ways  of  developing  and  maintaining  new  skills  A 
specific  goal  is  to  establish  aircrew  Simulator  Training  Requirements  by  the  application  of  new 
simulation  technologies  and  aircrew  training  system  designs  that  will  permit  savings  by  providing 
high  quality  training  in  cost-effective  ways.  The  objective  of  this  effort  is  to  develop  guidelines 
for  visual  systems  and  users.  The  present  effort  was  conducted  under  Work  Unit  1123  32  04. 
Simulator  Field-of-View  Requirements,  by  Capt  Kevin  W.  Dixon,  Principal  Investigator,  and  assisted 
by  iLt  Victoria  A  Rojas,  lit  Gretchen  M.  Krueger,  and  Capt  Luke  Simcix. 

An  oral  presentation  of  this  paper  was  given  at  the  1988  American  Institute  of  Aeronautics 
and  Astronautics  (AIAA):  Flight  Simulation  Technologies  Conference  in  Atlanta,  Georgia  by  Lt 
Victoria  A  Rojas. 
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EYE  TRACKING  DEVICE  FOR  THE  MEASUREMENT 
OF  FLIGHT  PERFORMANCE  IN  SIMULATORS 


I.  INTRODUCTION 

The  design  and  training  effectiveness  evaluation  of  simuia.jr  visual  systems  have  historically 
depended  upon  data  from  subjective  questionnaires  and  from  objective  pilot  performance 
measures.  The  fieid-of-view  (FOV)  size  is  one  important  characteristic  of  the  visual  system 
which  is  often  evaluated  using  either  or  both  of  these  methods. 

The  questionnaire  approach  uses  personal  opinion  data  to  evaluate  characteristics  of  the 
FOV  This  approach  has  been  used  in  a  number  of  studies  including  Newman  (1983).  Randle. 
Roscoe,  and  Petitt  (1980),  and  Wiekhorst  and  Dixon  (1987).  Although  this  method  is  widely 
used,  questionnaires  suffer  from  being  subjective  in  nature;  also,  they  give  no  indication  of  the 
portion  of  the  FOV  being  used,  and  they  do  not  reveal  where  attention  is  allocated  within  the 
visual  field  For  example,  an  evaluation  conducted  b  Goodyear-Red iffusion  (O’Neal,  1984)  for 
the  F-15  Visual  System  asked  48  pilots  to  perform  certain  tasks  in  a  160°  horizontal  (H)  X  60° 
vertical  (V)  limited-FOV  F-15  simulator,  rate  the  FOV  for  each  task,  and  estimate  additional  FOV 
requirements  for  tasks  in  which  the  FOV  was  rated  less  than  acceptable  The  results  of  this 
study,  based  on  pilot  opinion,  indicated  that  the  evaluated  visual  system  (160°  H  X  60°  V)  can 
substantially  enhance  air  superiority  operations  training  and  air-to-surface  operations  training 
Subjective  evaluations  performed  in  this  manner  contribute  a  large  amount  of  data;  however,  it 
is  difficult,  based  on  this  technique,  to  draw  conclusions  concerning  the  actual  performance  or 
visual  behavior  of  the  pilots. 

The  use  or  pilot  performance  measures  (aiiiiuue,  airspeed,  etc)  during  simulator  missions 
can  also  be  deceiving,  because  the  pilot  may  use  other  means  (instruments)  to  compensate 
for  the  lack  of  fidel  ty  of  the  visual  system.  For  example,  Nataupsky,  Waag,  Weyer,  McFadden, 
and  McDowell  (1979)  conducted  a  transfer-of-training  study  using  student  pilots  as  subjects  in 
the  T-37  Advanced  Simulator  for  Pilot  Training  (ASPT)  and  in  the  T-37  aircraft.  The  students 
learned  basic  contact  maneuvers  in  one  of  two  FOV  conditions  (300°  H  x  150°  V,  or  48°H  x 
36°V)  and  were  evaluated  on  these  maneuvers  during  their  first  aircraft  mission.  The  automated 
performance  measuring  system  and  instructor  pilot  ratings  were  used  for  analysis  during  the 
simulator  missions,  whereas  instructor  ratings  and  individually  recorded  flight  parameters  were 
analyzed  for  the  aircraft  sorties.  The  results  revealed  no  significant  differences  in  performance 
due  to  FOV  conditions. 

The  two  approaches  outlined  in  the  preceding  paragraphs  give  the  researcher  some  idea 
of  the  pilot's  impressions  and  of  the  pilot's  performance,  but  do  not  allow  objective  determinations 
of  the  pilot's  visual  activity. 

Continuously  recording  the  pilot's  eye  position  offers  a  data  collection  method  to  quantify 
pilot  visual  behavior  during  simulated  flight.  Eye-tracking  allows  direct  assessment  of  the  visual 
behavior  of  the  pilot,  and  an  indication  of  his  attention  allocation  at  any  time  during  the  mission. 
Use  of  an  eye  tracking  system  would  have  increased  the  understanding  of  why  the  pilots  in 
the  Goodyear  Rediffusion  study  rated  the  field  of  view  for  each  task  as  acceptable  or  unacceptable 
in  the  Nataupsky  et  al.  study  it  would  have  provided  information  as  to  whether  or  not  the 
student  pilots  changed  their  visual  strategy  during  the  smaller-FOV  condition  to  compensate  for 
the  lack  of  outside  visual  cues. 
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II.  SYSTEM  ARCHITECTURE 


After  a  determination  was  made  that  eye  position  data  would  fill  the  gap  between  subjective 
questionnaires  and  pilot  performance  measures,  a  suitable  system  to  collect  visual  behavior 
data  had  to  be  found.  The  major  factors  considered  were  cost,  ease  of  calibration,  comfort, 
free  head  movement,  and  transportability. 

The  system  chosen  was  a  Model  210  eye  movement  monitor  from  Applied  Science  Laboratories. 
The  Eye  Movement  Monitor  employs  a  photoelectric  sensing  and  processing  technique  to 
determine  magnitude  and  direction  of  eye  movements.  Eye  illumination  and  sensing  are 
accomplished  with  infrared  illumination  to  minimize  distraction  to  the  subject.  The  device  is 
attached  to  a  headband-mounted  camera  (see  Figure  1). 


The  instrument  is  capable  of  measuring  horizontal  eye  movements  over  a  range  of  approximately 
+  /-15  degrees,  with  an  accuracy  of  about  1  degree  and  a  precision  of  better  than  1/4  degree 
Vertical  eye  movements  can  be  measured  over  a  range  of  approximately  +/-15  degrees,  with 
an  accuracy  of  about  2  degrees  and  a  precision  of  better  than  1  degree  (see  Appendix  A, 
System  Specifications;.  The  device  can  contain  an  instantaneous  field  of  view  of  30  degrees 
horizontally  and  vertically,  and  a  full  360-degree  field  of  regard. 

The  video  fixation  point  capabilities  of  the  device  present  either  crosshairs  or  a  cursor 
superimposed  over  a  television  monitor  image  of  the  scene  being  viewed  by  the  pilot.  The 
visual  scene  is  broken  up  into  seven  areas  which  correspond  to  the  seven  window  locations 
The  image  is  captured  with  a  video  recorder  and  times  are  coded  to  complete  the  data  collection 
procedure.  The  last  component  of  tho  system  is  Tapsmaster,  a  video  software  program  which 
is  used  to  analyze  the  data  Tapemaster  captures  a  time  code  from  the  videotape  and  supplies 
a  definition  of  that  time  period  with  respect  to  the  scene  (window)  being  viewed.  These  data 
are  then  manipulated  to  provide  desciiptive  and  inferential  statistics.  Figure  2  is  a  diagram  of 
the  overall  system. 
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Figure  2.  Diagram  of  Eye  Tracking  System. 


III.  IMPLEMENTATION 

Preliminary  evaluations  were  conducted  to  ensure  component  compatibility  and  proper  system 
operation  Familiarization  and  proficiency  training  of  the  experimenters  occurred  in  a  static 
environment  for  the  practice  of  setup  and  calibration  techniques.  Each  experimenter  required 
approximately  15  hours  of  training. 

The  initial  field  test  of  the  system  was  accomplished  during  a  C-130  Weapon  System  Trainer 
(WST)  field-of-view  study  at  Little  Rock  AFB,  AK  (Dixon,  Martin,  Rojas,  &  Hubbard,  1988).  The 
C-130  WST  is  a  full  mission  simulator  which  provides  computergenerated  imagery  for 
out-of-the-window  visual  cues.  The  visual  system  produces  day,  dusk,  and  night  scenes  through 
a  six-window,  five-channel,  color  cathode-ray  tube  display  system  with  infinity  optics.  This  study 
investigated  the  effect  of  FOV  on  pilot  performance  for  low  .evei  flight  and  an  airdrop  in  the 
C-130  WST.  The  study  was  performed  using  two  different  FOV  configurations.  The  full-FOV 
condition  incorporated  all  six  windows  to  provide  a  visual  field  of  160°  H  by  35°  V.  In  the 
limited-FOV  condition,  the  forward  fout  windows  were  used  to  provide  a  113°  H  by  35°  V  visual 
field. 

Two  methods  of  data  collection  were  used  throughout  the  study.  The  first  method 
incorporated  the  eye  tracking  system  to  determine  whether  or  not  pilots'  visual  behavior  or 
performance  is  altered  in  the  different  FOV  configurations.  Automated  pilot  periormance  measures 
were  also  collected,  these  included  pilot  control  inputs  and  system  parameters.  Twelve  male 
C-130  pilots  with  a  crew  qualification  of  instructor  pilot  or  aircraft  commander  served  as  subjects. 
The  eye-monitoring  device  was  worn  by  the  pitots  for  the  duration  of  the  simulator  test  flight, 
which  lasted  approximately  25  minutes.  Subjective  questionnaires  concerning  the  eye  camera 
and  headband  were  given  to  each  pilot  at  the  conclusion  of  the  test.  The  pilots  experienced 


some  slight  discomfort  in  wearing  the  apparatus,  but  reported  that  it  did  not  restrain  head 
movement  or  interfere  with  the  mission 

Data  from  the  eye  position  camera  were  encoded  using  a  personal  computer  applications 
program.  This  program  (Tapemaster,  Comprehensive  Video  Supply  Corp.)  enabled  the  specification 
of  visual  area  codes  (area  within  each  window,  Instruments,  or  other)  for  the  visual  field  based 
on  the  video  fixation  point  (see  Figures  3,  4,  and  5).  "Instruments'  were  defined  as  eyes 
transitioning  to  the  instrument  panel,  and  'Other"  was  all  fixations  not  related  to  windows  or 
instruments.  The  definitions  for  each  area  were  manually  encoded  into  the  computer.  Once 
encoded,  the  data  were  transferred  to  the  VAX  11/780  for  further  analysis.  The  variables  used 
for  analysis  included:  time  in  each  window,  number  of  glances  in  each  window,  percent  of 
total  time  and  glances  for  each  window,  and  percent  of  time  per  glance.  The  results  of  the 
study  showed  a  significant  difference  in  visual  behavior  between  the  full-  and  limited-FOV 
conditions.  This  difference  occurred  in  the  percent  of  time  spent  looking  at  instruments,  the 
forward  window,  and  the  left  window  adjacent  to  the  forward  window.  During  the  full-FOV 
condition  (in  comparison  to  the  limited-FOV  condition),  pilots  spent  less  time  looking  at  instruments 
and  the  forward  window,  but  more  time  looking  in  the  left  peripheral  window  The  simulator 
performance  measures  Indicated  no  significant  differences  between  the  two  FOV  conditions. 
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Figure  5.  Example  of  C-130  Eye  Focal  Point  (Instruments). 


Experimental  etforts  relying  solely  on  simulator  performance  measures  would  have  concluded 
that  there  were  no  differences  between  the  two  FOV  conditions.  However,  the  visual  behavior 
differences  indicate  that  pilots  are  altering  their  visual  strategy  to  maintain  performance  parameters. 


A  second  field-of-view  study  was  performed  on  the  Simulator  for  Air-to-Air  Combat  (SAAC) 
located  at  Luke  AFB,  AZ.  The  objective  cf  this  study  was  to  determine  the  field  of  view  used 
in  specific  air-to-air  tasks,  free  engagements,  and  mutual  support  operations.  The  SAAC  consists 
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fight  against  each  other  or  against  the  computer  in  air  combat  engagements.  Data  on  number 
of  glances  in  each  window,  time  spent  on  instruments,  number  of  transitions  from  each  window, 
and  eye  position  relative  to  target  position  were  collected.  Final  data  analysis  is  currently 
underway,  and  the  results  will  be  published  at  a  later  date.  In  contrast  to  the  relative  stability 
in  the  C-130  simulator,  air-to-air  combat  maneuvers  require  a  great  deal  of  dynamic  head 
movement  The  incorporation  of  the  eye  tracking  system  in  this  environment  was  a  significant 
step  in  validating  system  versatility  and  tiaxibilily  of  the  eye  tracking  device 


IV.  APPLICATIONS 

Results  from  the  static  C-130  WST  the  dynamic  F-15/F-16  tests  of  the  eye-monitoring 
system  indicate  that  it  has  a  strong  potential  for  both  research  and  training  environments. 
Proposed  visual  training  research  applicatio  s  include  areas  In  workload  assessment  and  attention 
allocation  Many  of  these  applications  equire  initial  research  into  visual  behavior  and  its 
relationship  to  mental  processing 

Investigations  into  such  areas  as  pilot  cross-check  techniques  are  one  of  the  many  training 
applications  possible.  Based  on  comparisons  of  the  visual  strategy  of  individual  pilots  (focal 
point,  time  spent  on  each  instrument,  etc  ),  recommendations  for  cross  check  traininy  for 
inexperienced  pilots  could  be  made. 

The  eye  tracking  system  could  also  be  used  for  system  design  and  evaluation.  For  instance, 
a  quantitative  evaluation  of  the  visual  cues  used  in  iow-level  flight  can  be  obtained  with  the 
eye  tracking  system.  Such  an  evaluation  will  be  extremely  useful  in  order  to  optimize  the 
scene  content  in  visual  displays.  If  incorporated  into  research  and  design  methodologies,  the 
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eye  tracking  system  could  provide  valuable  information  on  the  eye  focal  point  for  various  tasks 
The  advantages  of  the  eye  tracking  system  make  it  a  valuable  tool  that  can  be  used  alone  or 
in  conjunction  with  subjective  and  objective  performance  measures  for  valid  and  reliable  decisions 
concerning  visual  systems. 


V.  LIMITATIONS 

ine  eye  tracking  system  does  have  its  limitations.  It  only  reveals  the  eye  focal  point  and 
does  not  account  for  visual  information  available  and  processed  from  the  visual  periphery  For 
example,  when  the  pilot  is  looking  in  a  forward  window,  his  periphery  is  also  being  s',  n  ated 
by  the  information  in  the  adjacent  window  Although  he  does  not  look  at  the  adjacent  side 
windows  as  frequently  as  the  forward  windows,  he  receives  input  from  these  windows.  The 
eye  tracking  data  do  not  reveal  this  added  information.  Thus  data  interpretation  is  limited  to 
information  within  the  system’s  visual  field  only. 

The  absence  of  a  fixed  reference  point  is  also  a  limitation  that  at  present  does  not  allow 
automation  of  the  data  acquisition  and  data  reduction  phases  of  the  eye  tracking  system  A 
fixed  reference  point  would  allow  a  moving  coordinate  system  to  be  superimposed  over  the 
visual  field.  This  coordinate  system  would  allow  for  automatic  processing  of  the  eye  position 
data. 

Data  reduction  of  the  recorded  event  is  a  manual  process  which  requires  approximately  2 
minutes  of  coding  for  each  minute  of  flight  simulation  The  coded  data  are  then  reduced  with 
a  computer  software  piogram  which  produces  a  table  of  descriptive  statistics.  The  total  process 
is  very  time-consuming  and  labor-intensive.  For  example,  it  took  a  team  of  three  people  45 
hours  apices  tc  produce  the  descriptive  statistics  for  a  study  with  rqq  trials 


VI.  SYSTEM  ENHANCEMENTS 

A  number  of  enhancements  are  planned  for  the  eye  tracking  system  These  improvements 
will  overcome  the  limitations  of  labor-intensive  data  reduction  and  lack  of  a  fixed  reference 
point. 

An  automated  eye  tracking  system  is  currently  being  designed  by  ISCAN,  Inc  ;  the  Air  Force 
Human  Resources  Laboratory  at  Williams  AFB,  AZ;  and  the  University  of  Dayton  Research 
Institute.  The  p** nosed  system  wili  automatically  calibrate  the  system  to  the  subject’s  eye  iouai 
point.  Synchronized  infrared  light-emitting  diodes  (LEDs)  will  be  placed  in  the  visual  field  and 
be  referenced  by  an  X-Y  tracking  system.  The  eye  focal  point  data  and  the  X-Y  tracking  data 
will  be  correlated  to  determine  exact  position  in  the  field  of  view.  Specially  designed  software 
will  formal  the  data  for  statistical  analysis.  Another  enhancement  is  the  use  of  a  wider  angle 
camera  to  encompass  more  of  the  pilot's  field  of  view. 

The  infrared  LEDs  and  the  X-Y  tracker  will  allow  for  a  reference  point  to  be  established  for 
automation  of  the  data  acquisition  and  data  reduction  phases. 
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